Toward Practice-Based Continuing Education Protocols: Using Testing to Help Physicians Update Their Knowledge.

INTRODUCTION: Using assessment to facilitate learning is a well-established priority in education but has been associated with variable effectiveness for continuing professional development. What factors modulate the impact of testing in practitioners are unclear. We aimed to improve capacity to support maintenance of competence by exploring variables that influence the value of web-based pretesting. METHODS: Family physicians belonging to a practice-based learning program studied two educational modules independently or in small groups. Before learning sessions they completed a needs assessment and were assigned to either sit a pretest intervention or read a relevant review article. After the learning session, they completed an outcome test, indicated plans to change practice, and subsequently documented changes made. RESULTS: One hundred twelve physicians completed the study, 92 in small groups. The average lag between tests was 6.3 weeks. Relative to those given a review article, physicians given a pretest intervention: (1) reported spending less time completing the assigned task (16.7 versus 25.7 minutes); (2) performed better on outcome test questions that were repeated from the pretest (65.9% versus 58.7%); and (3) when the learning module was completed independently, reported making a greater proportion of practice changes to which they committed (80.0% versus 45.0%). Knowledge gain was unrelated to physicians' stated needs. DISCUSSION: Low-stakes formative quizzes, delivered with feedback, can influence the amount of material practicing physicians remember from an educational intervention independent of perceptions regarding the need to engage in continuing professional development on the particular topic.

A student initiative to implement peer-led study groups for a pharmacogenomics course: Evaluation of student performance and perceptions.

INTRODUCTION: To better elucidate the impact of cooperative learning outside the classroom, a student-initiated research project was conducted to explore the effects of participating in peer-led study groups (PLSGs) on student examination scores and perceptions. METHODS: First-year pharmacy students were given the opportunity to participate in weekly PLSGs for a pharmacogenomics course during spring 2016 and spring 2017. Student exam performance was stratified by those who attended vs. those who did not. Optional pre- and post-course surveys examined student perceptions of PLSGs. RESULTS: No significant differences were seen between the attendance groups in spring 2016. In spring 2017, student attendees were significantly more likely to pass two of their six exams (p = .04, p = .0029) and to have higher exam scores on one exam (p = .02) in comparison to non-attendees. Overall exam score averages were significantly different between attendees and non-attendees during spring 2017 (p = .03) but not during spring 2016 (p = .38). Perception surveys indicated students believed participation helped them to demonstrate competency and build confidence. Additionally, students reported they felt more comfortable clarifying questions during the study groups vs. during class time. CONCLUSIONS: The impact of study group participation on student exam performance was minimal over the two years of data collection, but there were instances where exam scores were positively impacted. Students perceived value in study group participation even if it did not translate directly to improved exam performance on all exams.

A Computerized Functional Skills Assessment and Training Program Targeting Technology Based Everyday Functional Skills.

Today, many functional skills are technology-based, so development of a technology-based training program has broad importance. Here we present a computerized functional skills training program that was paired in half of the participants with a commercially available cognitive training (CCT) program. Non-impaired older individuals (NC) aged 60+ (n=45) and similarly aged individuals with mild cognitive impairment (MCI; n=50) were randomized to receive 12 weeks of twice-weekly computerized functional skills training (CFST) or 12 weeks of twice-weekly sessions split between CCT and CFST. Skills trained were use of an ATM; internet banking; ticket kiosk; telephone and internet prescription refill; medication management; and internet shopping. As with previous functional capacity assessments, we focus on completion time for each simulation. 51 participants completed the training program, either by mastering all 6 tasks (34) or completing 12 weeks of training. 44 more participants completed 4 or more training sessions so they were also analyzed for improvement up to their last training session. Completion time for all 6 tests significantly improved from the baseline assessment to the final training session in both groups of participants (all p<0.001 with an average improvement in task completion time of 45%). Further, there was no differential improvement in MCI and NC in the 6 tests from baseline to end of training (all t<1.66, all p>0.12). Finally, combined CCT plus CFST did not differ from CSFT alone on any of the percent-change score measures (all t<1.64, all p>0.11). Both NC and MCI groups evidenced substantial improvements in performance. CCT supplementation led to similar functional gains with half as many training sessions. The NC participants proceeded through the training fairly rapidly even without CCT supplementation; MCI participants required more training but learned equivalently. These findings suggest that even in cases with memory impairments, functional skills can be efficiently learned with training.

General Surgery Resident Use of Electronic Resources: 15 Minutes a Day.

BACKGROUND: General surgery resident performance on the American Board of Surgery In-Service Training Exam (ABSITE) has been used to predict American Board of Surgery (ABS) passage rates, selection for remediation programs, and ranking of fellowship applicants. We sought to identify electronic resource study habits of general surgery residents associated with successful test scores. STUDY DESIGN: A single-institution, retrospective review of general surgery resident use of 2 electronic study resources, Surgical Council on Resident Education (SCORE) and TrueLearn (TL), were evaluated for the 7 months before the 2019 ABSITE. Metrics included TL question performance, SCORE use, and a survey about other reading sources. These metrics were evaluated in 3 ABSITE percentile groupings: ≥80th, 31st to 79th, and ≤30th. RESULTS: The ≥80th and 31st to 79th percentile groups scored higher on TL questions, at 69% and 67.7%, respectively, compared with 61.4% for the ≤30th percentile group (p < 0.03). The ≥80th percentile group spent on average 14.6 minutes/day on SCORE compared with 5.0 minutes/day and 4.7 minutes/day for the 31st to 79th and ≤30th percentile groups, respectively (p < 0.04). The ≥80th percentile group spent 34.8 minutes/session (77 sessions) compared with 19.2 minutes/session (49 sessions) and 20.7 minutes/session (43 sessions) in the 31st to 79th and ≤30th percentile groups, respectively (p = 0.009). CONCLUSIONS: Our nomogram incorporates time spent accessing an electronic content-based resource, SCORE, and performance on an electronic question-based resource as a novel method to provide individualized feedback and predict future ABSITE performance.

Diagnostic readiness tests: Preparing nurse practitioner students for national certification examinations.

BACKGROUND AND PURPOSE: Nurse practitioner (NP) faculties are challenged to offer programs that balance didactic knowledge with the clinical experiences required to prepare graduates for the complexities of health care. Students have the stress of extrapolating the information they obtain during these programs and applying it to postgraduation certification examinations. Innovative educational tools are necessary to ease the uncertainty that NP students experience to graduate and pass certification examinations. Tools include academic-clinical partnerships (ACPs) to enhance clinical opportunities and resources to integrate diagnostic readiness tests (DRTs) into curricula. METHODS: This was a quasi-experimental one group, pretest/posttest design using a convenience sample of NP students randomly assigned to clinical in ACP or non-ACP clinical placements. They completed the DRTs twice during their final program semesters. CONCLUSIONS: The overall DRT group scores for the NP students significantly improved over time. The Wilcoxon signed ranks test showed that the posttest scores were significantly higher than the pretest scores. The Mann-Whitney U test revealed no differences between the ACP and non-ACP students. Students performed the best on assessment, diagnosis, management, and pharmacology domains. This demonstrates support for the integration of DRTs into NP programs. IMPLICATIONS FOR PRACTICE: Integrating DRTs into NP programs can facilitate transition to the NP role.

Timing is key to providing modified assessments for students with specific learning difficulties.

INTRODUCTION: Medical students who are diagnosed with a specific learning difficulty (SpLD) will typically receive a reasonable adjustment within examinations in the form of modified assessment provision (MAP). This study investigated whether the timing of SpLD diagnosis and subsequent implementation of MAP has an impact on performance in applied medical knowledge multiple choice question (MCQ) assessments. METHOD: The MCQ performance of 108 students diagnosed with SpLD who received a MAP was monitored and compared with 1960 students who received an unmodified assessment, over 5 years of a medical program. Students who received a SpLD diagnosis in the latter years of the program were identified as not receiving a MAP in assessments prior to diagnosis. RESULTS: Differences were found between declaration and diagnosis, with 44.4% of students who declared and 48.1% who did not declare subsequently receiving a diagnosis. Students with SpLD who receive a MAP increase their applied medical knowledge assessment performance, although there is a delay of up to a year for this impact to reach significance. CONCLUSION: Early diagnosis of SpLD is necessary to ensure the intended benefit is received from MAP.

Comparison of tests to evaluate the vertical rebound jump in basketball players / Comparação de testes para avaliação do ressalto vertical em jogadores de basquetebol

ABSTRACT This study aimed to compare the vertical rebound jump performance of basketball players by using Hurdle Jump and Drop Jump tests based on different performance parameters, in addition to investigating whether the height of the hurdles used (barrier and box) are equivalent when the optimal and maximum heights for the tests mentioned are determined. Thirteen players of a male basketball team were included in this study (age 18.77 ±1.78 years old, height 182 ±7.97 m, body mass 76.4 ±9.55 kg, practice time 6.0 ±1.8 years). Data normality and homogeneity were tested by applying Shapiro-Wilk and Levene tests, respectively. The paired t-test was used to compare the means of the results obtained in the tests; the significance level was p ≤ 0.05. The results regarding the comparison of the performance parameters means were the following: the maximum vertical rebound jump and reactive strength index did not show significant differences (p = 0.256 and p = 0.243, respectively); on the other hand, differences were seen for the corresponding obstacle heights. These differences can be attributed to both, individual characteristics of each subject and the specificities of the tests.

RESUMO O objetivo do presente estudo foi comparar o desempenho do ressalto vertical de jogadores de basquetebol por meio dos testes Hurdle Jump e Drop Jump a partir de diferentes parâmetros de desempenho e verificar se a altura dos obstáculos utilizados (barreira e caixote) se equivalem quando forem determinadas a altura ótima e a máxima para os testes mencionados. Participaram do estudo 13 jogadores de uma equipe de basquetebol masculina (idade 18,77 ±1,78 anos, estatura 182 ±7,97 centímetros, massa corporal 76,4 ±9,55 quilogramas, tempo de prática 6,0 ±1,8 anos). A normalidade e a homogeneidade dos dados foram testadas por meio dos testes Shapiro-Wilk e Levene, respectivamente. O Test T pareado foi utilizado na comparação das médias dos resultados obtidos nos testes e o nível de significância foi p≤0,05. Os resultados das comparações das médias dos parâmetros de desempenho: ressalto vertical máximo e índice de força reativa, não apresentaram diferenças significativas (p=0,256 e p=0,243, respectivamente), em contrapartida, foram identificadas diferenças nas alturas dos obstáculos correspondentes. Essas diferenças podem ser atribuídas às características da individualidade de cada sujeito e às especificidades dos testes.

Academic accommodation strategies for pharmacy students with learning disabilities: What else can be done?

INTRODUCTION: Colleges and schools of pharmacy in the US are faced with the challenge of providing appropriate academic accommodations for students with learning disabilities (SLD) including those with dyslexia, dyscalculia, or dysgraphia. Although there are no population statistics about SLD in pharmacy schools, current data show that their number at higher education institutions across the US has increased. This commentary discusses how the rigorous pharmacy school curriculum affects SLD and proposes creative and innovative measures to overcome these obstacles. PERSPECTIVE: The shift in the public perception about SLD from a state of stigma to a manageable condition in recent years has greatly contributed to increase the number of SLD enrolled in higher education institutions. Although most colleges and universities can now offer some type of academic accommodation to SLD compared to few years ago, other academic accommodation recommendations to SLD remain challenging to implement, especially for SLD enrolled in pharmacy programs. IMPLICATIONS: SLD enrolled in pharmacy programs could benefit from a multifaceted approach involving the school's office of disability services, medical practitioners, behavioral and language specialists, as well as some sort of curricular changes in order to be academically successful and become practice ready upon graduation.

A web-based review game as a measure of overall course knowledge in pharmacotherapeutics.

BACKGROUND AND PURPOSE: Implementation of games is common in pharmacy education, but limited information exists about the relationship between students' game performances and course grades. Our study was designed to determine if scores on a comprehensive web-based review game correlate to overall pharmacotherapeutics course and course series grades. EDUCATIONAL ACTIVITY AND SETTING: Two cohorts of students in Pharmacotherapeutics IV, the last course in a four course series, were administered a web-based quiz review game (Kahoot!) at the end of the course. Student performance on the game was compared to grades throughout the Pharmacotherapeutics course sequence to determine if the games were reflective of the students' performances. All students enrolled in Pharmacotherapeutics IV in 2016 and 2017 were included in the analysis. Pearson correlation was performed on the scores from the review game compared to the grades in the pharmacotherapeutics course series. FINDINGS: A total of 197 students, 111 in the 2016 cohort and 86 in the 2017 cohort, were included in the analysis. The correlation coefficient (r) for the review scores and Pharmacotherapeutics IV course grade was 0.399 and 0.461 for the 2 cohorts (p < 0.001). Almost all component comparisons between the review scores and the other pharmacotherapeutics course grades were also significantly correlated (p < 0.01). SUMMARY: Student performance on the review game significantly correlated with pharmacotherapeutics course grades. Review games are fun tools to review course content and can serve as an effective method to determine student understanding, progression, and knowledge.

What Patients Want in a Smartphone App That Supports Colonoscopy Preparation: Qualitative Study to Inform a User-Centered Smartphone App.

BACKGROUND: The preparation for colonoscopy is elaborate and complex. In the context of colorectal cancer screening, up to 11% of patients do not keep their colonoscopy appointments and up to 33% of those attending their appointments have inadequately cleansed bowels that can delay cancer diagnosis and treatment. A smartphone app may be an acceptable and wide-reaching tool to improve patient adherence to colonoscopy. OBJECTIVE: The aim of this qualitative study was to employ a user-centered approach to design the content and features of a smartphone app called colonAPPscopy to support individuals preparing for their colonoscopy appointments. METHODS: We conducted 2 focus group discussions (FGDs) with gastroenterology patients treated at the McGill University Health Centre in Montreal, Canada. Patients were aged 50 to 75 years, were English- or French-speaking, and had undergone outpatient colonoscopy in the previous 3 months; they did not have inflammatory bowel disease or colorectal cancer. FGDs were 75 to 90 min, conducted by a trained facilitator, and audiotaped. Participants discussed the electronic health support tools they might use to help them prepare for the colonoscopy, the content needed for colonoscopy preparation, and the features that would make the smartphone app useful. Recordings of FGDs were transcribed and analyzed using thematic analysis to identify key user-defined content and features to inform the design of colonAPPscopy. RESULTS: A total of 9 patients (7 male and 2 female) participated in one of 2 FGDs. Main content areas focused on bowel preparation instructions, medication restrictions, appointment logistics, communication, and postcolonoscopy expectations. Design features to make the app useful and engaging included minimization of data input, reminders and alerts for up to 7 days precolonoscopy, and visual aids. Participants wanted a smartphone app that comes from a trusted source, sends timely and tailored messages, provides reassurance, provides clear instructions, and is simple to use. CONCLUSIONS: Participants identified the need for postcolonoscopy information as well as reminders and alerts in the week before colonoscopy, novel content, and features that had not been included in previous smartphone-based strategies for colonoscopy preparation. The ability to tailor instructions made the smartphone app preferable to other modes of delivery. Study findings recognize the importance of including potential users in the development phase of building a smartphone app.

Language Matters: Development of an Objective Structured Language Test for Foreign Physicians - Results of a Pilot Study in Germany.

Objective: To develop a scientifically sound and standardized medical language examination for the State of Bavaria according to the requirements set forth by the 87th Conference of State Health Ministers. This Sprachtest für Ausländische Mediziner (SAM, Language Test for Foreign Physicians) ought to become part of the licensing procedure for foreign physicians in Germany. Using testing stations that are situation-based, it will assess medical language competence and communication skills at the proficiency level of C1. Methods: Case scenarios for four mini-interviews of 10 minutes each were developed. For the written part of the exam, consisting of two separate testing stations with a combined duration of 40 minutes, one video of a physician taking a patient's history and one annotated set of laboratory results were developed. Based on the analysis of existing scientific literature as well as real-life examples, features and characteristics of professional medical language were identified. This served as the basis for the development of itemized rating scales for each of the testing stations. The exam was validated in three simulated trial runs. Each run was video-recorded and subsequently graded by a team of test-raters. Results: 19 participants took part in the three trial runs. A benchmark (gold standard) could be set for 18 of these. A ROC-analysis yielded an AUC-value of .83. This confirmed the predictive quality of the SAM-test. The reliability of the SAM-test could be calculated for only ten participants. The internal consistency, calculated with the use of Cronbach's Alpha, was .85. The pass/fail mark was calculated based on the Youden-Index and yielded a result of >60%. Conclusion: The SAM-test presents a statistically valid medical language examination with a high level of objectivity. As required, it tests language proficiency at the level of C1 and uses authentic communication scenarios within a standardized test setting. Additional studies with larger test samples will help to further validate this test and thus guarantee a higher degree of reliability.

Testing anxiety in undergraduate medical students and its correlation with different learning approaches.

OBJECTIVES: Undergraduate medical students experience a considerable amount of stress and anxiety due to frequent exams. The goal of the present study was to examine the development of exam related anxiety and to test for a correlation between anxiety and learning approaches. METHODS: A whole class of 212 medical students was invited to participate in the study. During their first term, trait anxiety and learning approaches were assessed by use of the state-trait-anxiety inventory (STAI-T) and the approaches-and-study-skills-inventory-for-students (ASSIST), respectively. Acute state anxiety was assessed twice in the course of the second term. To that extent, the STAI-S in combination with measuring salivary cortisol were employed immediately before two oral anatomy exams. RESULTS: Our most important results were that a surface learning approach correlated significantly with anxiety as a trait and that students with a predominantly strategic approach to learning were the least anxious yet academically most successful. CONCLUSION: As surface learners are at risk of being academically less successful and because anxiety is a prerequisite for burn-out, we suggest that medical faculties place particular emphasis on conveying strategies for both, coping with stress and successful learning.

Experience with implementing pyramid examinations in an elective pharmacy course.

BACKGROUND AND PURPOSE: This article describes the implementation of pyramid tests, an assessment strategy that utilizes individual and cooperative testing for formal examinations in an elective course for third year pharmacy students. This strategy provides individual accountability and uses group collaboration to extend students depth of thinking through essay responses. Pyramid tests have been suggested as a way to improve grading efficiency compared to traditional tests that use essay questions. EDUCATIONAL ACTIVITY AND SETTING: A two-credit hour pharmacy elective taught by synchronous video conferencing on two campuses of a large public university utilized pyramid tests for both the midterm and final examinations. Each examination had individual and group components and utilized a simple metacognitive question. FINDINGS: The averages from the individual portions of the pyramid tests were similar to the prior year, while the combined group (2/3 weight) and individual (1/3 weight) pyramid test scores averages were slightly higher than the previous year's average (87% compared to 86%, respectively). Student feedback on this assessment technique was generally positive. SUMMARY: This assessment strategy was an efficient way to test students' knowledge. Although it had a collaborative component, it also provided individual accountability. Time spent on course administration and grading was similar to previous years that employed fixed choice assessments without collaborative work.

Students as stakeholders in assessment: how students perceive the value of an assessment.

INTRODUCTION: For assessment to fill an educational role, students must see the results generated by assessment as valuable, and actively engage with this feedback in order to support learning. Few studies include examinees as stakeholders in validation beyond general notions of acceptability. Here, we explore students as stakeholders in the validation of a newly implemented assessment. METHODS: A student-relevant validity framework based on the unified theory of validity was created and adapted to a survey format. Likert-style items were used to examine first- and second-year medical students' perceptions of a new cumulative assessment, with several open-ended items. Analysis included: mean ratings per subscale of validity evidence, thematic analysis of comments, and a correlation between questionnaire subscores and exam performance. RESULTS: Seventy-seven students participated (20.5%). Student perceptions of the assessment were favourable, with significantly different ratings across validity evidence (Response Process (4.8 (SD = 0.7); scored/6), Content (4.6(0.9)), Consequential (4.4(0.8)), Internal Structure (4.2(0.9)), and Relationship to Other Variables (4.0(1.0))). Exam performance correlated with subscores for Relationship to Other Variables (r = 0.34, p < 0.005) and Response Process (r = 0.24, p < 0.05). DISCUSSION: Students perceived the assessment as facilitating learning, providing 'checkpoints', and were disappointed when it did not meet their expectations regarding the purpose of assessment. If students perceive that results do not reflect their future performance in clinical environments, or do not align with their perceived purpose of assessment, the educational value of assessment may be limited. It is critical to understand when, and how students engage in interpreting and integrating assessment-generated feedback to ensure that assessment contributes positively to learning.

Analysis of question text properties for equality monitoring.

INTRODUCTION: Ongoing monitoring of cohort demographic variation is an essential part of quality assurance in medical education assessments, yet the methods employed to explore possible underlying causes of demographic variation in performance are limited. Focussing on properties of the vignette text in single-best-answer multiple-choice questions (MCQs), we explore here the viability of conducting analyses of text properties and their relationship to candidate performance. We suggest that such analyses could become routine parts of assessment evaluation and provide an additional, equality-based measure of an assessment's quality and fairness. METHODS: We describe how a corpus of vignettes can be compiled, followed by examples of using Microsoft Word's native readability statistics calculator and the koRpus text analysis package for the R statistical analysis environment for estimating the following properties of the question text: Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (Grade), word count, sentence count, and average words per sentence (WpS). We then provide examples of how these properties can be combined with equality and diversity variables, and the process automated to provide ongoing monitoring. CONCLUSIONS: Given the monitoring of demographic differences in assessment for assurance of equality, the ability to easily include textual analysis of question vignettes provides a useful tool for exploring possible causes of demographic variations in performance where they occur. It also provides another means of evaluating assessment quality and fairness with respect to demographic characteristics. Microsoft Word provides data comparable to the specialized koRpus package, suggesting routine use of word processing software for writing items and assessing their properties is viable with minimal burden, but that automation for ongoing monitoring also provides an additional means of standardizing MCQ assessment items, and eliminating or controlling textual variables as a possible contributor to differential attainment between subgroups.